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ABSTRACT 

Publishers of brief, time-valued information for engineers will be 
provided the means to publish their material online. Engineers in 
numbers suitable for statistical analysis will be provided with a 
means of reading this online material as well as with some simple 
information management tools. Data on the usefulness of the sys- 
tem will be gathered both by pretests and posttests administered 
to the subject and a control group and by real time recording of 
the engineers 's behavior online. 

The computer support will be the Online System developed at SRI 
over the last 14 years, which includes information retrieval, pub- 
lication, and monitoring functions and sophisticated communication 
facilities in an integrated system. The publications available to 
the engineers will include SRI's Datalog abstract service and a 
time-valued engineering publication such as Aerospace Daily. 
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PROJECT DESCRIPTION 

Summary 

Overview 

This proposal responds to NSF Program Solicitation NSF 76-51 "Im- 
proved Understanding and More Productive Use of Scientific and 
Technical Information (STI)." It addresses Category 1, 
Innovations in Engineering Publication. 

We propose a cooperative project between SRI (the project team's 
source of technological innovation and analytic expertise), a com- 
mercial publisher of information for engineers, and at least two 
separate organizations that employ engineers who will receive the 
information and who will be subjects of the experiment. 

We will secure the cooperation of companies that employ engineers 
and that are customers both of SRI's Datalog abstract service and 
a commercial publication that disseminates time-valued information 
to engineers. At each organization we will identify engineers who 
are interested in the experience of computer access to informa- 
tion. We will provide the participating engineers with CRT 
(televison-like) terminals and training in a simple set of com- 
mands that will. allow them to scan titles and ask to see the en- 
tire item when they are interested, to send telegram-like mes- 
sages, and to establish a list of key words which the system will 
use to automatically exclude items not of interest to each indi- 
vidual. 

In addition to the services discussed above, we will provide the 
publishers with training in the use of the computer system and an 
appropriate work station. 

Researchers will base their evaluation of this project upon infor- 
mation gathered from two sources: surveys and automatic monitor- 
ing of the engineers in action on the computer system. An exis- 
ting SRI software facility will monitor and record the engineers 
as they work. We will administer pretests and posttests to the 
engineers to determine the usefulness of the system to them, their 
change in attitudes, and the like. The information gathering and 
analysis will take into consideration already existing studies on 
the use of Datalog and the SRI information handling services in- 
volved. 
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Background 

The Computer System 

NLS (the Online System) is proposed as the computer medium for all 
aspects of this experiment. It is a general-purpose system for 
handling and communicating textual information. NLS is highly ap- 
propriate to support this study because it combines facilities to: 

Publish documents 

View information flexibly on a cathode ray tube 

Monitor the activities of users 

Bind together scattered users 

Tailor commands to applications 

Publication 

Complete facilities for creating, editing, and^ formatting printed 
documents exist in NLS. 

CRT Viewing - 

The small size of CRT screens is a serious handicap to browsing 
and fluid reading in many computer-based information systems. NLS 
goes a long way toward overcoming this problem, and related prob- 
lems of maneuvering in information space, by providing the user 
with many ways to display selected, evocative parts of informa- 
tion. An example related to this proposal is the ability for a 
user to display only headings, only first lines, or only para- 
graphs that contain given words and then to move freely among such 
views and full views. 

Monitoring Users 

A monitoring program permits us to record exactly what NLS users 
do when they work on the computer and creates record files suit- 
able for statistical manipulation. 

Binding Scattered Users 

NLS has proved highly successful in various efforts that include 
organizationally and geographically scattered individuals such as 
the engineers, publishers, and researchers who will be involved in 
the proposed project. The tools of cooperation include access to 
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common files, electronic mail which may be recorded and cataloged 
for recall, and simultaneous display and control of the same work 
material on different user's screens. 

Tailored Commands 

A software mechanism, called the Command Meta Language, makes it 
easy for programmers working with NLS to create commands that pro- 
vide selected functions to users. For a simple application like 
the activities of the engineers in the proposed study, it is pos- 
sible to provide an effective set of commands that is easy to use 
and easy to learn. 

Datalog 

SRI's entire Datalog will be entered into the computer. Datalog 
is a monthly bulletin, containing abstracts of recently completed 
SRI studies, that is sent to industrial and other organizations 
who subscribe to SRI's Long Range Planning Service (LRPS). (See 
attached sample in Appendix C.) The subscribers are made up of 
national and international companies in the Fortune 500 class, 
most of which have heavy engineering orientations. It is 
presently published by typists who prepare camera-ready copy manu- 
ally, and is disseminated by U.S. mail. SRI has kept extensive 
statistics on the subscribers ' use of the. publication since the 
beginning of the Datalog program. These statistics on the past 
use of Datalog will provide a rich basis for comparison with its 
use when it is distributed via computer. 

A Commercial, Time-Valued Publication 

To widen the range of information involved and to provide practi- 
cal experience that will interest and stimulate the publications 
industry, SRI has sought cooperation from a commercial distributor 
of time-valued information to engineers. Although the 
uncertainties of the time frame have prevented any commercial 
publisher from making a firm commitment, a keen interest has been 
expressed by Kenneth Koppel of Ziff-Davis, who oversees several 
such publications. Mr. Koppel is interested in participating in 
this project mainly because he anticipates the growth of electron- 
ic publication and wants ground -floor experience. 

Our conversation with Mr. Koppel has centered on Aerospace Daily, 
a typical time-valued publication. Aerospace Daily (see Appendix 
D) is six to eight 650-word pages long and is published daily in a 
one-column magazine format. It is presently published by typists 
who create camera-ready copy the evening before it is to be dis- 
tributed by U.S. mail (at considerable disadvantage to many West 
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Coast users). The subscribers are generally Fortune 500 class 
companies. Aerospace Daily is expensive; and a typical organiza- 
tion has a single subscription which goes to descision makers upon 
arrival, and is later circulated to engineers. 

Response To Current Issues 

The proposed project will result in a prototype information dis- 
semination system that may be of commercial interest, a computer- 
based test bed for further research on how people use information, 
and reports which should prove stimulating to research on the dis- 
semination of technical information. 

The services to be offered will respond directly to several prob- 
lem areas in the use of engineering publications. Published in- 
formation will be made more timely as the delay between publica- 
tion and actual receipt by the engineers is reduced through use of 
the computer-based communication system. The filtering devices 
will enable engineers to use their reading time more efficiently. 
The ability to locate information quickly will be improved by ap- 
plying automatic search routines to the information stored in the 
computer and by facilitating the collection and filing of informa- 
tion. Publications will become more valuable as two-way communi- 
cation between authors and, readers is mac^e easy. 

The proposed study should also provide a basis for progress in two 
problem areas in research on dissemination of technical and scien- 
tific information. It has been widely recognized that engineers 
depend on word of mouth to learn much more than do scientific 
researchers [2]. Although the usefulness of word of mouth systems 
has been elucidated, they have many drawbacks [1]. This system 
will allow engineers to seek information in a way that is neither 
paper shuffling nor word of mouth. 

The facilities for monitoring how the subjects seek information is 
of interest beyond the problems specific to engineering informa- 
tion. Until now, it has been extremely expensive or impossible to 
monitor people's information seeking as they do it, and our knowl- 
edge in this field has been restricted to what we could find out 
in after-the-fact surveys, interviews, and the like, or through 
report sheets checked by the user. It is now possible to gather 
this information in detail in real time, without conscious activi- 
ty by the subject. 

Continuity with SRI Interests 

SRI welcomes NSF's program solictation because it offers us a 
chance to synthesize activities in several areas of continuing in- 
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terest to our staff. As detailed in Organizational Background, 
several departments in SRI have long-term commitments to the areas 
of specialization involved in the proposed study, such as elec- 
tronic aids to publication, information science, office automa- 
tion, and various strategies for increasing productivity in 
intellectual work. 

Organization of this Proposal 

This proposal closely follows the structure of the Program 
Solicitation. The technical proposal is divided into sections ac- 
cording to the proposal format described by the solicitation. The 
longest section is the Project Description which discusses, point 
by point, the requirements spelled out in the third paragraph of 
Category 1. Actual cost figures appear in a separate cost volume. 
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Hypotheses 

An important set of criteria for evaluating STI services is con- 
cerned with the usefulness of the services as perceived by the 
user population of engineers. Do these services improve their 
ability to locate and obtain relevant information? Is the speed 
of the delivery of time-sensitive information increased? Is it 
easier to file, reach, and retrieve published information? 

In the paragraphs below these questions are related to a number of 
hypotheses. Verifying these hypotheses will constitute the major 
part of the evaluation effort. The experimental methodology to be 
used is outlined in a subsequent section of this proposal. In ad- 
dition to questions directly related to our hypotheses, users will 
be asked about the value of the services, the strengths and 
weaknesses of the delivery system, desirable additional features, 
and related questions. 

Information use and communications behavior of scientists and en- 
gineers have been studied extensively. Many hypotheses may be 
drawn from the body of literature. We have selected hypotheses, 
and referenced some of the relevant research, that most closely 
correspond to the goals of this program solicitation for 
exploration in our proposed experiment. A brief comment on each 
reference is included. The hypotheses are summarized below. 

Hypotheses: 

Hypothesis 1 : The delay between the publication and reading of 
technical news letters and periodicals will be reduced through 
the use of electronic information systems which automatically 
alert engineers to the existence of items of interest and make 
those items conveniently available. 

Hypothesis 2: Engineers will find the timely availability of 
information more useful. 

Hypothesis 3: Key word filtering of online information will 
increase the personal relevance of an information stream. 
Here, relevance is defined as the ratio of items judged to be 
of interest to the number of items scanned. 

Hypothesis 4: Engineers will make greater use of automatic 
filters over time as they gain confidence that the filters are 
effective. 

Hypothesis 5: The usefulness of periodic literature as a 
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source of information will increase because online retention of 
abstracts permits easy search, retrieval, and personal filing. 

Hypothesis 6: The asynchronous message capability of the 
abstract delivery system will be perceived as beneficial. 

Hypothesis 7: The ability for any user to immediately attach a 
comment to any online article will result in: 

a) greater contact between users with similar interests, and 

b) more direct communication between users and publishers. 

Hypothesis 8: Engineers will use the economic and business in- 
formation contained in Datalog as much as the scientific and 
technical information. (The publication contains about an 
equal mix of each.) 

Discussion and Related Research 

Timeliness of Information. There is a considerable delay between 
the composition of periodic technical literature and its perusal 
by engineers. Contributing factors are the delay between the com- 
pletion of an author's original composition and the availability 
of printed copy, and the significant delay between printing and 
eventual reading by the engineer. One aspect of this proposal is 
directed at reducing the second of these: the time spent in 
mailing and routing technical literature. 

Hypothesis 1 : The delay between the publication and reading of 
technical news letters and periodicals will be reduced through 
the use of electronic information systems which automatically 
alert engineers to the existence of items of interest and make 
those items conveniently available. 

Hypothesis 2: Engineers will find the timely availability of 
information more useful. 

Supporting Reference: 

(Parker, 1970) 

Surveys and interviews of the users of an STI service (SPIRES) 
unanimously emphasized the importance of timeliness. [391 



Information Overload. An engineer suffers from information 
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overload partly because he has no mechanism for screening out 
irrelevant information in the literature he examines. 

Hypothesis 3: Key word filtering of information stored in the 
computer will increase the personal relevance of an informa- 
tion stream. Here, relevance is defined as the ratio of items 
judged to be of interest to the number of items scanned. 

Hypothesis 4: Engineers will make greater use of automatic 
filters over time as they gain confidence that the filters are 
effective. 

Supporting References: 

(Hall, Clague, and Atchison, 1972) 

Scientists who use selective dissemination of information (SDI) 
services tend to value them despite a general decline in 
valuation among other scientists. A familiarity effect is sus- 
pected. [25] 

(Scott, 1959) 

Dissimilarity of styles in a group is important to high perfor- 
mance. [45] . 

(Wolek, 1970) 

The phenonenon of information overload is a limitation on our 
ability to process information. This is a function of the com- 
plexity of the information received. [52] 



Storage And Retrieval. Storage and retrieval of information in 
periodicals are currently hampered by the need to pass on the 
original or return it to a library and by the difficulty of filing 
the same article under several headings. Photocopying is an 
expensive and inconvenient solution. Because future needs can not 
always be predicted, information thought to be irrelevant may be 
discarded and then later be found valuable. 

Hypothesis 5: The usefulness of periodic literature as a 
source of information will increase because online retention of 
abstracts permits easy search, retrieval, and personal filing. 

Supporting References: 
(Allen and Gerstberger, 1967) 
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Accessibility, familiarity, and ease of use are more important 
than quality in determining which sources of STI are used 
first, although not in determining sources of information from 
which selection will be made. [4] 

(Friedlander, 1973) 

Ease of accessibility promotes use. [17] 

(Gerstberger and Allen, 1968) 

Ease of use and familiarity affect what source is used first, 
although not necessarily which information source provides the 
information a scientist or technologist ultimately uses in his 
or her work. [20] 

(Rosenberg, 1967) 

Accessibility and ease of use determine which source of infor- 
mation will be used first. [43] 

Interaction and Communication. Interaction between users and 
publishers of abstracting and information services is hampered by 
shortcomings of the telephone and mail service. Relatively long • 
publishing cycles make' it very difficult to maintain any type of 
dialog resulting from a published aritcle. Thus, articles having 
the potential to act as catalysts for the discovery of people with 
common interests or for the beginning of fruitful discussions 
often fail to achieve these results. 

Hypothesis 6: The asynchronous message capability of the 
abstract delivery system will be perceived as beneficial both 
by publishers and users. 

Hypothesis 7: The ability for any user to immediately attach a 
comment to any online article will result in: 

a) greater contact between users with similar interests, and 

b) more direct communication between users and publishers. 



Subjects of Interest to Engineers. Research indicates that engi- 
neers seek as much economic and market information as scientific 
and technical information. This point has not been well estab- 
lished and more concrete information would certainly be valuable 
to publishers of materials for engineers. 
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Hypothesis 8: Engineers will use the economic and business in- 
formation contained in Datalog as much as the scientific and 
technical information. (The publication contains about an 
equal mix of each.) 

Supporting Reference: 

(Thompson, 1975) 

Engineers use economic and business information to an equal ex- 
tent with scientific and technical information. [493 
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Administration 

Delivery System 

Interactive computer terminals designed to display textual infor- 
mation will be installed in two engineering organizations where 
they will be readily accessible to the sample groups of engineers 
and engineering managers. A terminal will also be installed at 
the publishing office for "Aerospace Daily" in New York (or at the 
office of another similar publication). These terminals will be 
connected via dial telephone lines and the ARPA Network to a com- 
puter and additional terminals at SRI. 

A general purpose office automation and publication software sys- 
tem, called NLS, is available on the SRI computer. This computer 
system allows textual material to be entered at any terminal and 
read electronically at any other, and it provides the tools neces- 
sary to implement the study proposed here. The system includes a 
message capability that will allow the participants in the study 
to exchange information, and provides a method for the 
experimenters to monitor the engineer's information seeking activ- 
ities. We propose to employ NLS to deliver the two services that 
will aid engineers in their access to and use of published infor- 
mation. 



Services 

Service Number 1 

The Ziff-Davis Publishing Company has expressed a keen interest in 
participation in this project. The company publishes a number of 
widely read services for engineers. It is aware of the growing 
demand for electronic delivery, and one of the Ziff-Davis publica- 
tions, such as "Aerospace Daily," will be placed online by the 
publisher. Aerospace Daily is widely read throughout the aero- 
space industry. It contains time-sensitive information such as 
notice of major Federal policy decisions and contract awards. It 
is several pages long, simply formatted, typewritten, and contains 
no graphics, photos, or ads. Each news item is one paragraph 
long. (See sample. Appendix D) 

For the proposed project, the news items will be entered into the 
computer when the Aerospace Daily editorial staff approves them 
for release, or when the newsletter is printed. They will be 
available instantly to engineers in our sample population. Each 
engineer on the computer distribution list will be able to estab- 
lish his own key word and/or key phrase filter in the NLS comput- 
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er. This filter will screen the publication automatically. Vflien 
the engineer begins a new work session, the system will list by 
title only those items that have passed through the filter. A 
simple command will allow the engineer to read the entire item. 

This automatic filter should help eliminate the irrelevant news 
items the engineer usually must check through before he finds 
items relevant to his particular interests. The process will also 
short circuit delay caused by the routing lists within the engi- 
neer's organization often attached to copies of publications such 
as Aerospace Daily to reduce copying or extra subscription costs. 

Whenever an engineer reads the complete text of an item, his iden- 
tity tag will be attached to that item and will function as a key 
word. A simple command will allow the engineer to search all old 
items by key words. If he uses his own identity tag in the 
search, he can recover all items he read previously. Although the 
engineer has the option of using his tags solely for his own (and 
the experimenter's) information, he may instruct the computer that 
other users in his organization or in the whole user community 
will be able to view his tags or use them for key word searches. 
This will provide a means for identifying a community with common 
interests. All items will be recorded for the duration of the 
experiment. ) 

Any user may easily and simply direct a ^telegram-like message to 
the publisher or to other users. 



Service Number 2 

SRI's entire "Datalog" will be entered into and distributed via 
the computer. Datalog is a monthly bulletin, containing abstracts 
of recently completed SRI studies, that is sent to industrial and 
other organizations who subscribe to SRI's Long Range Planning 
Service (LRPS). (See attached sample in Appendix C. ) The 
subscribers are made up of national and international companies in 
the Fortune 500 class, most of which have heavy engineering 
orientations. SRI has kept extensive statistics on the 
subscribers' use of the publication since the beginning of the 
Datalog program. These statistics on the past use of Datalog will 
provide a rich basis for comparison with its use when it is dis- 
tributed via computer. Normally these time-sensitive bulletins 
are sent by mail. 

Making Datalog accessible through a computer will accomplish sev- 
eral things. First, delivery will be much faster. Second, the 
publication can be processed through individualized profile 
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filters to reduce irrelevancy. Third, they will be stored in the 
computer to facilitate filing and searching through back issues. 
Finally, interaction between user, publisher, and author will be 
made easy. 

Because Datalog is part of a larger service offered by SRI, 
clients may send messages to SRI seeking further details about any 
item mentioned in the bulletin. Thus, the NLS two-way message ca- 
pability will provide a communications channel with distinct prac- 
tical advantages over both the telephone or letter mail. The 
telephone obviously requires simultaneous availability of both 
parties. In practice, several days and several tries are 
sometimes needed to make contact, and the telephone cannot provide 
information in fileable form. Letters are slow, formal, and 
expensive. The ability of NLS to conveniently convey 
asynchronous, fileable messages fills a gap between phone calls 
and letters, and is particularly useful when several cycles of 
interaction are needed to satisfy a certain request. 



Implementation of Services 

As mentioned above, Ziff-Davis, the publisher of Aerospape Daily, 
has expressed a strong interest in participating in this 
experiment. Its responsibility will be to enter Aerospace Daily 
or possibly a similar publication into the computer using a 
terminal loaned under this project. SRI's Long Range Planning 
Service has already agreed to enter the Datalog and perhaps se- 
lected source articles. 

Although we have discussed the idea with Boeing and Ford Aero- 
space, no definite arrangements have been made with engineering 
groups willing to receive these services and serve as subjects for 
the evaluation. We believe it will be fairly easy to find two 
suitable engineering groups that subscribe both to Datalog and a 
publication such as Aerospace Daily. To limit travel and communi- 
cation costs, we will seek organizations in the San Francisco Bay 
area whenever possible. We will require that each participating 
organization help us identify a subject and control group of engi- 
neers, cooperate in the conduct of the experiment (including 
pretesting and posttesting) , and donate time for SRI to train one 
or more of the people in the subject group in the operation of the 
terminal. This individual will either operate the terminal for 
the engineers or assist them in doing so. 

Terminals will be supplied to engineers, publishers, and SRI 
researchers according to a staggered schedule based on need. (See 
"Schedule Budget" below. ) A day or two of training in NLS as ap- 
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propriate will be provided for one person at each engineering site 
and for the publisher's staff who will use the system. In addi- 
tion, brief training will be provided to engineers for using the 
simple information handling tools. 

Datalog and engineering companies within a local call of SRI will 
have direct access to the SRI computer. If necessary, we propose 
access to NLS computing facilities operated by SRI via the ARPA 
Network. Use of the ARPA Network reduces communication costs to 
the price of a phone call to the nearest ARPA Network node. The 
operators of the ARPA Network, the Defense Communication Agency, 
have accomodated use of the network for various NLS-based 
experiments, including two by the National Science Foundation, and 
we anticipate no difficulties in this matter. 

Access to computer power will be through purchase of SRI-KL Com- 
puter Utility service from SRI as described in the sections 
entitled "Budget" and "Schedule." A number of government agen- 
cies, including NSF, presently subscribe to this service, and con- 
tractual arrangements are well established. 



Administration of Experiment 

Within- the participating engineering organizations, subjects will 
be selected by canvassing for engineers interested in partici- 
pating in an online experience, as discussed in the section 
entitled "Research Design." 

For the engineers' use, SRI will program a simple command inter- 
face which will allow them to login easily and identify them- 
selves , and create and update a key word profile of their 
interests. The system will list titles of new items of interest 
based on their individual profiles and allow them to display the 
full text if they choose. It will also enable them to send 
telegram-like messages to publishers as well as to each other and 
allow them to identify their specific interests publically or 
within their business organizations. 

All the activities of the engineers will be recorded, command by 
command, for analysis, as described in the section entitled "Re- 
search Design." 
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Research Design 

This section details the research design for evaluating the impact 
of a real time, interactive publication system upon engineers and 
their use of STI. Because many factors will contribute to the im- 
pact of the system, we are suggesting several kinds of measure- 
ment . 



Summary 

The experimental design will be based on comparisons between the 
pretests and post tests of an experimental group and a control 
group. The independent variable is the use of the service. As an 
important enhancement to, and verification of, this testing, sta- 
tistics providing an ongoing record of the behavior of system 
users will be automatically collected (without disturbing the 
users). The experimental design will guarantee users rights to 
privacy: no measurements will be made without each user's knowl- 
edge and consent. Both quantitative and qualitative measures will 
be employed to determine differences resulting from the use of the 
service and to measure exogenous variables. 

Confounding effects well known in this kind of field study — for 
example, selection, maturation, and Hawthorne effects— will be ad- 
dressed. Exogenous variables such as demographics, job assign- 
ments of the subjects studied, and physical and personal environ- 
ments will also be measured. This data may be structured for 
correlation, analysis of variance, and factor analysis (as appro- 
priate). 



Experimental Design and Analysis 

The field experiment will be as rigorous as possible in a non- 
laboratory setting, with the resources available. Matched groups, 
an experimental and a control group, will be selected from the or- 
ganizations willing to participate in the experiment. The 
experimental group will use the online service for a continuous 
test period of approximately eight months. Service usage, 
attitudes, and STI usage will be measured in pretests and 
posttests, while actual use of the system will be monitored online 
throughout the period of the experiment. The control group will 
be given a placebo treatment to counteract Hawthorne effect. 
Exogenous factors will be measured and the extent to which they 
are causal variables will be determined. The data will be ana- 
lyzed to determine whether the basic hypotheses listed above are 
supported or must be rejected. 
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There is no known method for measuring, quantitatively, the abili- 
ty of persons to assimilate information into knowledge that is 
relevant to their needs. Therefore qualitative measures are nec- 
essary and will be balanced carefully against the quantitative. 

Three kinds of qualitative and quantitative instruments will be 
used: (1) automatically monitored usage data, (2) attitude 
ratings (including relevance ratings), (3) and surveys (including 
job task type profiles and demographic profiles). The 
quantitative instruments include computer-collected transaction 
and usage data, such as connect time, commands used, computer re- 
sources used, and data bases accessed. Variables such as 
demographic factors, job task descriptions, information seeking 
behavior descriptions, training, and experience will be described 
and categorized for correlation analyses. 

The validity of the hypotheses will be examined using standard 
statistical techniques applied to the quantitative and qualitative 
data collected both online through the system and offline in the 
questionnaires. At the same time, a thorough description of 
events and settings not measured will ensure that nothing is over- 
looked because of an overemphasis upon statistics. 



Exogenous Factors 

In any field study numerous factors other than the independent 
variable, such as individual differences, selection, and 
maturation, can influence outcomes. Six categories of exogenous 
factors are of special importance in this study: (1) population 
characteristics, (2) training and effectiveness, (3) equipment 
type and availability, (4) Hawthorne effect, (5) data base value, 
and (6) environment. 

1. Population characteristics. A population will be selected 
from those members of participating organizations who respond 
favorably to a selection survey questionnaire about the proposed 
service. The population size will satisfy statistical 
significance requirements and be economically feasible. Half of 
the subjects will be allocated a placebo treatment and will not 
require online support. It will be explicit in this study that 
the service has an effect on the kind of person who chooses the 
service. Results may differ from those derived from experiments 
in which the subjects are randomly selected and required to use 
the service; the conclusions will be so adjusted. 

Matched samples will be selected using the sample selection sur- 
vey, based upon demographic characteristics and job assignments. 
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This latter criterion must be examined carefully, since the par- 
ticular tasks of a worker greatly determine his information needs. 

2. Training and effectiveness. A priori experience with computer 
technology in general, and STI systems in particular, will be 
surveyed in the pretest. Training time will be recorded from 
questionnaires, and/or system-maintained usage records. User ef- 
fectiveness will be judged by both the trainer and the subjects 
independently. 

3. Equipment type and availability. The type of terminal used 
and the availablity of the terminal and system will be noted, es- 
pecially since previous studies have emphasized the importance of 
proximity of a service to its usage. Connect time to the system 
will be recorded automatically. 

4. Hawthorne effect. The Hawthorne effect will be very strong in 
this experiment, caused by novelty, awareness that one is partici- 
pating in an experiment, an environment modified by trainers, 
surveyors, and special procedures, and knowledge of results at 
different points. The length of the eight month experimental pe- 
riod will permit partial control of this effect. In addition, au- 
tomatic monitors will capture usage data without disturbing the 
users. • 

A placebo will be administered to the control group during the 
experimental period to cause effects similar to the treatment. 
The placebo will include interviews and surveys of the information 
needs, and must not include any new STI information or services 
that would change the control group's performance. 

5. Data base value. The data bases selected for this experiment 
will have different levels of value for each subject. The selec- 
tion of the population based upon their interest and need as 
expressed in the selection survey will ensure a level of need suf- 
ficient for the use of the system. 

6. Environment. Factors in the organizational and physical envi- 
ronments, such as management attitudes, terminal proximity, and 
visual and acoustical characteristics of the terminal area, will 
have an effect on the service usage. 

In addition to the exogenous factors of special concern in this 
study, the design may account for others, such as history, 
maturation, testing, instrumentation, statistical regression, and 
experimental mortality, that generally affect the validity of 
experimental design [28]. 
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Quantitative Measures and Collection 

Quantitative data will be gathered where possible through the use 
of automatic monitors that are currently part of the NLS software. 
Data collection from the control group will be from 
questionnaires . 

NLS provides a monitoring facility whereby data can be collected 
and maintained on almost every parameter of system operation, and 
in particular a subject's online use of the system. The programs 
also permit tabulation of the data in user-specifiable formats for 
analysis. If this facility is monitoring a user, it records her 
login time, her use of CPU, each occasion when NLS responds to a 
command word, recognizes an address, completes or aborts a com- 
mand, each occasion when a user backspaces out of a command word, 
types in some text (but not the text itself), changes her view of 
the file, specifies something on the screen with the NLS cursor 
control, or asks for online help. It also records certain error 
messages from the system, invalid characters typed by the user, 
the total load on the computer sampled from time to time, and some 
other housekeeping information about the state of the computer and 
the software. 

With this system it would be possible to reconstruct in detail the 
activity of a user at a given session of the proposed system, ex- 
cept the content of messages sent. The information is stored in 
special files constructed both to save space and to provide the 
basis of easy statistical manipulation of the results. 

Quantitative measures obtained through system records are used to 
explore some of our hypotheses. The following discussion and 
table list the quantitative measures, sources of the data, inter- 
vals of collection, units of measure, and hypotheses to which the 
measures pertain. 

Timeliness. The period between the origination of information and 
receipt by the end user will be measured both before the 
experimental service and during its full operation. 

Relevance. Relevance is the ratio of items retrieved (the user 
peruses them) to the items judged relevant. It is by definition a 
subjective metric, although it yields a quantity that has been 
found to be valid in other studies [5]. Immediately following 
each session using the system, the subject will be asked to rate 
the relevance of each item retrieved. 
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Retrieval -m. Each manual retrieval of a publication in any of the 
data bases will be recorded. 

Requests. The number of data base interrogations made after the 
delivery of items selected by each subject's dissemination profile 
will be recorded. 

Prof ile-chge . Each change to the online SDI profile maintained 
for each user will recorded. 

Retrieval -o. Online retrievals will be recorded automatically by 
the monitor, including an identification of the user. 

Back ref . The number of searches of the online data bases con- 
taining previously issued items will be recorded. 

Contact type. If through-system contacts are made, the sender and 
recipient will be recorded, along with the number of transactions. 

Comment. Each use of the comment facility will be counted, in- 
cluding the subject originating the command and the destination. 

Info-type. The number of online accesses and the subjects 
accessing each data base will be recorded. 

Cost-1 . The cost of each session will be computed, including a 
breakdown of labor support and computer resources. 

Cost-2. Based upon cost-1 and publishing costs, the long term 
costs of the service will be computed, including projections for a 
service continuing beyond the experiment. 

Logins. The number of times each subject uses the system will be 
tabulated during the experimental period. 

Connect. The actual time spent online will be automatically 
tabulated by the monitor. The time connected to the system will 
be logged-in hours per subject. 

Review-t. A description of the subject's time spent reviewing 
literature through the system will be recorded from the Job Task 
Profile survey (pretest) and the posttest survey. This is a 
subjective estimate on the part of the subject. 
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Table of Measures 











APPLICABLE 


METRIC 


SOURCE 


INTERVAL 


UNIT 


HYPOTHESIS 


timeliness 


quest 


pre /post 


days 


1 


relevance 


quest 


periodic 


ratio 


3 


retrieval -m 


tabulate 


occurrence 


count 


3 


requests 


monitor 


ongoing 


count 


4 


profile-Ghge 


monitor 


ongoing 


count 


4 


retrieval -o 


monitor 


ongoing 


count 


6 


back ref 


monitor 


ongoing 


count 


6 


contact 


monitor 


occurrence 


proximity 


7a 


comment 


monitor 


ongoing 


transaction 


7b 


info type 


monitor 


ongoing- 


data base 


, 8 


cost-1 


compute 


ongoing 


^/session 


»« 


cost-2 


compute 


ongoing 


$/days 


»« 


logins 


monitor 


ongoing 


count 


«« 


connect 


monitor 


ongoing 


time 


«» 


review-t 


quest 


pre/post 


time 


»« 



NOTES : 

Pre/post — Pretests and posttests. 

Ongoing — Cumulative data that will be presented for various units 
of time (e.g., cost/hour). 

Compute—The value will be computed from accumulated data. 

Quest — Data will be accumulated from survey questionnaires. 
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Monitor — An automatic collection and tabulation of data wherever 
the computer is used. 

**— These may have indirect relevance to the hypotheses that may 
appear, for example, in factor analyses of the data. 
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Survey Questionnaires 

Questionnaires must be relied upon to collect data that cannot be 
quantitatively measured, either because in the field test setting 
there is no automatic monitor of the specific process, the re- 
sponse desired is subjective, or it is too expensive to gather the 
data more directly. Several questionnaires will be given in pack- 
ages to minimize obtrusiveness and facilitate delivery. They in- 
clude the population sample selection survey, the pretest 
questionnaires for determining demographic characteristics, job 
task descriptions, and the comprehensive posttest questionnaire 
directly addressing the effect of the experimental treatment. 

Three types of questions will be included in the surveys: 

1. Questions that provide a five-position scale to assess the ef- 
fect of the system on specific daily information seeking routines. 

2. Multiple choice questions. Redundant questions are included 
for internal validity checks. 

3. Open-ended response questions will be minimized. However, 
items such as comments on the forced choice questions that enrich 
the data will be incorporated into the case record. 



Population Sample Selection Survey 

The initial population for this survey should be between two to 
three times the size of the desired sample. The study population 
will be those respondents most interested in the experimental ser- 
vice and STI in general. The experimental group will be selected 
using stratified random sampling, which will match the groups on 
the basis of demographic characteristics. 

This brief survey will describe the STI service and ask if the 
respondent would use it. The effort to use the system will be 
clearly outlined so that the engineer understands what will be re- 
quired of him. 



Population Characteristics Survey 

This survey data may be used to aid the selection of a representa- 
tive sample of the engineers in the sample organizations. It will 
include age, sex, geographical location, education, native 
language, etc. It will be combined with the Job Task Type Survey. 
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Job Task Type Survey 

An understanding of how the sample group allocates work time will 
clarify the group's need for information and their use of the sys- 
tem. In this pretest survey the experimental and control groups 
will determine the percentage of time spent on each job task type. 
For a list of job task types, see Appendix A. 



Posttest Survey Questionnaire 

A survey will be administered at the end of the experimental peri- 
od to determine the specific reactions to use of system by the 
experimental group. This includes questions on information avail- 
ability, level of usage, kinds and sources of information most 
often utilized, the type of terminal, system availability, effect 
on workload, disappointments encountered, and particular uses. 



Attitude Change Measurement 

It is proposed to measure attitude shift, not straight attitude 
toward the technology. The comparison will be between the pretest 
and posttest, with the control group comparison serving to isolate 
effects caused by maturation. The attitude test will incorporate 
an instrument design [14, 46, 50], in which the rated items corre- 
spond to dimensions describing information behavior (see Appendix 
B). Statistical tests for reliability and internal consistency 
will be used. 

Certain methodological assumptions are made regarding the measure 
of attitude shift. 

1. If the service has an effect (positive or negative), then that 
effect will result in a corresponding measurable change in the 
attitudes of users toward the service, the general technology that 
it represents, and the STI they receive. 

2. Measuring the attitude of the population involved is one valid 
way of ascertaining the effects. Attitude change is strongly af- 
fected by self perceived effectiveness, and conversely, attitude 
strongly influences effectiveness. A priori attitude will affect 
the propensity to use the service. 

3. Respondent reports are a valid means of ascertaining changes 
in performance/effectiveness. Although subjective, a user's 
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judgements about the utility of a tool are meaningful when there 
is a significant number of users. 
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Potential Value 

The proposed system will give us a carefully measured example of 
timely, filtered information delivered online to engineers. If 
the service proves attractive to the engineers and publishers and 
economic analysis is favorable, it can continue as a commercial 
enterprise which will make more information easily available to 
engineers and will enable them to receive the time-valued informa- 
tion more rapidly. (See "Service "below. ) 

Regardless of the commercial viability of the service, the rapid 
transmission of time-valued, filtered information in a thoroughly 
metered environment gives an unusual opportunity to learn more 
about the information needs and information gathering patterns of 
engineers. Substantial literature has been written on the infor- 
mation gathering of research scientists [35], including 
descriptions of many sophisticated methods of filtering informa- 
tion that is delivered to the researchers [38]. Unlike the previ- 
ous studies, the information delivery proposed here is quick, ori- 
ented toward engineers, and allows detailed recording of what the 
subjects do with the information they receive. 

It is widely accepted that engineers learn what they need to know 
rather differently from research scientists, mostly by word of 
mouth [3], with attending methodological difficulties in studying 
what happens. The description of the word of mouth process pre- 
sented by Allen [1] ten years ago remains the model for research 
in this area. Current research by Gellam [19] may show some of 
the reasons why engineers work in this way, but it does not offer 
alternative methods for more efficient ways of working or more ef- 
ficient ways of studying the work patterns. 

We offer here an alternative method that is neither word of mouth 
nor paper shuffling. If, as we anticipate, it alters the behavior 
of the subjects, the unique ability of NLS to track the subjects 
as they read will enable us to develop a body of novel information 
on their behavior. 
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Service Extension 

This experiment should provide sufficient knowledge to determine 
the feasibility, cost, and effectiveness of the delivery system 
and the services described above. Placing a terminal in the of- 
fice of the publisher should enhance the possibility of having the 
publisher continue the service after the end of the experiment. 
There is no institutional or technical reason why the publishers 
involved could not continue to use and expand these services on a 
continuing, commercial basis [34, 36]. 

Uncertainties such as cost of the delivery system make it diffi- 
cult to be explicit about future extensions of the service at this 
time. More definitive plans for continuing evolution and imple- 
mentation will be developed during the experiment as appropriate. 
The system is undergoing technical improvements and hardware cost 
decreases that should improve the cost effectiveness materially 
during the next two years. 

It should be noted that the computer system, NLS, proposed for use 
in this study is available directly from SRI, from a commercial 
service bureau, and as a software system suitable for running on a 
user's (e.g., publisher or EPC) in-house computer hardware. 

Ziff-Davis is interested in this experiment primarily because it 
foresees the day when its service will be electronic and would 
like to get experience in this form of publication. We imagine 
that other similar publishers will feel this way, and this atti- 
tude will be even more widespread in two years. The combination 
of publisher interest, technical feasibility, and the improving 
cost picture should make it possible for the service built here in 
protot3rpe to flourish on its own. 



Month 5: Administer attitude pretest to both groups 
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Final Report 

SRI will prepare a final report describing the innovative distri- 
bution medium, the hypotheses tested, the results of the test, and 
reporting broadly the effect of the use of interactive communica- 
tion on the engineers involved. The report will include both the 
results of statistical analysis of data from the surveys and 
automatated monitoring of system use, for example, data 
transformed for correlation, variance analysis, factor analysis 
(if appropriate), and case studies that explore processes, such as 
technology transfer and behavioral inertia. 

The report will be available both online to users of NLS and 
printed in the conventional manner. We also anticipate that our 
results will be suitable for journal publication. We believe the 
publication of information about the tools used and the results of 
our research on the patterns of use by the engineers will serve as 
a stimulus to the publishing industry that serves engineers. 
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MANAGEMENT 



Organizational Readiness 

SRI is unusually well qualified as an organization to support this 
project. It is one of the nation's most highly respected research 
organizations, with a long record of successful completion of sim- 
ilar projects for the National Science Foundation and other 
clients. In particular, SRI has been a center for study both of 
innovative publications methods and dissemination of scientific 
information. (See Institutional Information below. ) An ongoing 
community of people committed to both topics is available here in 
an organizational setting with longstanding success at supporting 
such projects. 



Direction and Coordination 

The principal investigator in this study will be Elizabeth K. 
Michael, who will direct and coordinate an interdisciplinary team 
drawn from the resources of SRI. She will be responsible to the 
National Science Foundation for successful completion of the study 
on time, in budget, and with excellent research and reporting. 
Ms. Michael has directed the preparation of this proposal and in 
particular is responsible for the adminisfrative, budget, and 
schedule portions. We are fortunate that she will be available 
for this work since the project she currently leads, the closely 
related Document Production and Control System Design Study con- 
tracted by the Rome Air Development Center, will end about the 
time this work begins. 



Schedule 

The schedule shown below lists only high level tasks in a rough 
time frame. A schedule listing tasks in detail and more precisely 
specifying time for each will be prepared during the first month 
of the project. 

Month 1 : Preliminary talks with Datalog and Ziff-Davis clients to 
get a better idea of environments, potential number of 
participants, communication facilites, etc. 

Month 1 : SRI work station installed 

Month 1 : Computer service on SRI-KL commences at 1 CRU level 
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Month 1 : Prepare detailed work schedule 

Month 1 : Orientation and some training for BIP management 

Month 1 : Talk with publisher 

1. Select the publication, possibly Aerospace Daily 

2. Get letter of intent from the publisher to NSF 

3. Obtain NSF approval of the publication 
Month 1: Survey design 

1 . Population Sample Selection Survey 

2. Population Characteristics Survey 

3. Job Task Type Survey 

4. Posttest Survey Questionnaire 

Month 1 : System orientation for SRI BIP management and planning 
for the experiment 

Month 2: Order leased line for publisher 

Month 2: Order work station equipment 

Month 3: Design user interface for engineers 

Month 3: Program user interface for engineers 

Month 4: Administer Population Sample Selection Survey 

Month 4: Install work station at publisher's site 

Month 4: System orientation for publisher's management and plan- 
ning for the experiment 

Month 4: Train publisher's staff 

Month 4: Trial publication put online at the publisher's and de- 
livered to SRI 

4: On-going client and publisher support 

5: Administer attitude pretest to both groups 



Month 
Month 
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Month 5: Administer the Job Task Type Survey 

Month 5: Select the study population and divide study population 
into matched groups, control and experimental 

Month 5: Train BIP secretary 

Month 5: Publish Datalog 

Month 5: Collect statistics on Datalog customer contacts and re- 
quests for both online and offline editions 

Month 6: Begin administering the placebo for the control group 
(survey and interviews) 

Month 6: Install work station at Client 1 site. 

Month 6: Train Client 1. 

Month 6: Client 1 uses system for one month 

Month 6: Make any indicated revisions to software and training 
for subsequent clients. 

Month 6: ~ Begin online monitoring of system use for the 
experimental group 

Month 7: Computer service on SRI-KL increased to 2 CRUs 

Month 7: Install work station at site of Client 2 

Month 7: Train client 2 

Month 7: Client 2 uses the system 

Month 16: Administer the posttest survey to the experimental 
group 

Month 16: Adminster attitude posttest to the control and 
experimental groups 

Month 16: Analyze the collected data 

Month 17: Write the final report 
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Month 18: Deliver report, including recommendations for technology 
transfer to publishing industry 

Grant Ends 
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Budget Support 

The proposed detailed budget and all supporting schedules are in 
Part II of this proposal. A budget summary in NSF format is in- 
cluded in Part I in the following section. 

This budget represents an effort on SRI's part to make available 
economically a very large array of highly sophisticated computer 
and communications tools. Development of the equivalent tools 
would require on the order of 100 person-years of programming, not 
including the documentation and the background knowledge of user 
training and technology transfer necessary to this application. 

Approximately two thirds of the proposed budget is the provision 
for the services of senior professional research staff. The bulk 
of the remaining third is for leased work station equipment for 
the widely distributed participants. A relatively small amount is 
for computer use, communications costs, travel, etc. 



Cost Sharing 

The cost sharing assumed by SRI comes about from the fact that 
both-Datalog and a commercially published engineering daily news 
letter will be processed twice during the research period, once by 
traditional methods and again in the online computer environment. 

The total saving to NSF of this cost sharing will be approximately 
$15,500. This estimate is based on personnel costs for typing and 
editing nine issues of Datalog and 270 issues of a daily engineer- 
ing news letter, plus management orientation, planning, and coor- 
dination in both organizations. 
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INSTITUTIONAL AND BIOGRAPHICAL INFORMATION 

SRI's Background 

General Capabilities of SRI 

SRI is an independent, nonprofit corporation that performs a broad 
spectrum of research, development, and services under contract to 
business, industry, and government. SRI's work is directed toward 
problem solving as well as research in the abstract. The staff of 
SRI numbers over 3,000. More than 500 SRI staff members hold 
Ph.D. degrees, over 600 hold Master's degrees, and approximately 
800 hold Bachelor's degrees. Facilities at SRI's main offices in- 
clude extensive data processing, library, and laboratory support. 
The comprehensive technical libraries are well supplied with 
literature in the fields of document generation and dissemination 
of scientific and technical information. 

Research operations at SRI are organized into seven divisions 
representing major disciplinary fields. Overall supervision of 
research is vested in the Office of Research Operations which re- 
ports directly to the Office of the President. Both formal and 
informal arrangements of long standing exist to facilitate 
interdisciplinary research and development among ^the divisions and 
their subgroups. Work related to proposed the "Innovations in En- 
gineering Publications" project has taken place in the Management 
and Economics Division and in the Information Science and Engi- 
neering Division. 

Management and Economics Division 

We believe it is important to mention capabilities within the Man- 
agement and Economics division here, although no member of that 
staff is specifically proposed for the initial team. Besides the 
general relevance of their work, their knowledge of business ap- 
plication of electronic aids to publishing should prove very use- 
ful in the analysis of the commercial use of the system. The Man- 
agement and Economics Division is concerned with solving problems 
for business, industry, and government through the application of 
modern management and systems techniques. The division has over 
twenty years of experience in bridging gaps in effectiveness and 
efficiency and keeping client management current with useful, 
practical systems that support all aspects of management from re- 
search and development through operation and maintenance. They 
emphasize studies of the interaction of technology and business 
and the resultant problems of managing technological change. 
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The Electronic Industries Department 

The Electronic Industries Department has conducted extensive re- 
search in the area of office automation from the marketing and 
technical viewpoints as well as the motivational and organization- 
al perspectives. During the course of their multi-client study- 
covering business equipment, which had 30 sponsors, several hun- 
dred personal interviews were conducted, supplemented by over 
5,000 mailed questionnaires covering the fields of word proces- 
sing, printing equipment related to data processing, and small 
business computers. In addition, they have conducted a number of 
proprietary studies on a wide variety of office automation pro- 
ducts and markets. 

Information Systems Management Department 

This department provides management assistance in the area of 
business electronic data processing and related planning and man- 
agement. The department has extensive experience in the develop- 
ment and implementation of techniques and methods to improve the 
effectiveness of electronic data processing (EDP) requirements 
planning, systems development, computer operations, equipment uti- 
lization, EDP cost benefits, and systems analysis. They have 
helped many organizations plan their overall systems development, 
personnel training, project management', and EDP organization de- 
velopment. 



Information Science and Engineering Division 

The activities of the Information Science and Engineering Division 
are carried out in several laboratories and research centers in- 
cluding three that are especially relevant to the proposed study: 
the Information Sciences Laboratory, the Telecommunications 
Sciences Center, and the Augmentation Research Center. Each of 
the laboratories is composed of a number of groups with 
complementary interests and skills. 



Information Science Laboratory 

In general, ISL personnel plan, design, and develop computer sys- 
tems for business and industry, and carefully mold them to meet 
management and operational needs. Current and long-term company 
goals, user requirements, corporate growth factors, functional EDP 
needs, applicable technology, and cost effectiveness are taken 
into account, so that the system will become an integral part of 
the organization and enhance the productivity of its users. 
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Their objective is to provide clients with the complete services 
essential to the achievement of a responsive, growth-oriented com- 
puter system. Actively involved in concept formulation, design, 
implementation, and evaluation of computer systems, ISL has en- 
gaged in the development of large and small-scale data processing 
systems in a variety of fields, including publishing and informa- 
tion dissemination. 

ISL work spans a wide spectrum of activities, from the preparation 
of performance specifications to the actual implemention of sys- 
tems, with a dominating principle being the pragmatic synthesis of 
hardware, system software, and application software to form an in- 
tegrated system that meets specific goals of their customers. 

An important example of ISL's continuing interest in computer- 
based aids to publication is the National Science Foundation's 
Editorial Processing Center (EPC II) project. For NSF, ISL is de- 
veloping a system to support academic journal production based on 
input at scattered sites connected to a central computer by tele- 
phone lines. 



Telecomunications Science Center 

Typical projects in this center, that relate to this project 
include: 

Evaluation of Experimental and Operational Teleconferencing Sys- 
tems. The objectives of this research, conducted for the National 
Science Foundation, were (1) to bring together in one document all 
the currently available information on audio, video, and computer 
teleconferencing systems that have been set up around the world 
for either experimental or operational purposes, and (2) to pre- 
sent this information in a primer or handbook form so that it can 
be used by government agencies and others as an aid to decision 
making. The kinds of information being gathered include: system 
costs (start-up and operations); purpose; transmission capabili- 
ties (voice, video, facsimile, text, and so forth); number of lo- 
cations; user characteristics; effectiveness; and usage and 
evaluative data. 

Evaluation of FORUM Teleconference on Travel /Communication Trade- 
offs. This was a study of a recent international computer 
teleconference on the subject of travel /communication trade-offs. 
This conference, held over a two-week period in February 1974, in- 
cluded some 30 participants from the United States, Canada, and 
England. The conference was one of several taking place via the 
NSF-funded FORUM teleconferencing system. 
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Assessment of the conference was a coordinated effort of SRI, Bell 
Canada, and the Institute for the Future. Among SRI's 
contributions to the document was a description and analysis of 
communication patterns among the participants. 

Study of Regulatory and Policy Problems Presented by 
Interdependence of Computer and Communication Facilities. This 
study for the Federal Communications Commission investigated the 
growing interdependence of the computer and communications 
industries. As part of the study, it was necessary to estimate 
the extent of that interdependence by studying the trends in de- 
mand for telecommunications services through the use of data com- 
munications devices. This is one of the major uses of domestic 
telecommunications services to be expected in the next ten years. 



Augmentation Research Center 

The Augmentation Research Center develops and applies a broad- 
based computer support system that improves effective use of human 
capacities in connection with textual information in a variety of 
forms. The following brief description of some of the accomplish- 
ments of ARC over the past 13 years will attest to its leading po- 
sition in the development of effective services for people working 
with textual information-. . 

Early explicit recognition of the potential that online computer 
and communication technologies have in areas outside of straight 
numeric or accounting computation in enhancing the effectiveness 
and efficiency of managers, scientists, technical writers, engi- 
neers, programmers, and their supporting staffs in their daily 
work. 

Participation in the implementation of the ARPA Network, a nation- 
wide network connecting over 3,000 remote terminals to 150 differ- 
ent computers. 

Early explicit recognition of the importance to system building of 
an integrated system of text handling and system building tools. 

Pioneered the two-dimensional text work to be the foundation of an 
intelligent terminal system, and developed many highly interactive 
tools and concepts for working and browsing in information space 
that is characteristic of engineers and scientists looking for in- 
formation. 

Pioneered input device and work station design. 
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Pioneered in high-quality formatted publication of hard copy- 
through line printers, typewriters, photocomposition, and computer 
output to microfilm. 

Pioneered the concept of an integrated coherent workshop of many 
information tools with a uniform user interface. 

First with a comprehensive system for electronic mail control, 
addressing distribution, delivery, individual and group identifi- 
cation, cross -linking, and indexing. 

Over 100,000 hours of hands-on console experience with the use of 
NLS technology in daily work at a variety of sites. 

Recognition of the importance of integrating into the system 
building process mechanisms for studying and facilitating technol- 
ogy transfer, including establishment of special user interfaces, 
training, and other application support services. 

For the last three years, the Augmentation Research Center has 
offered a system of aids for information-based work through a 
time-sharing service, the ARC Utility. About 15 government and 
private organizations supporting about 300 users now subscribe. 
Their activities include both publication,, such as Air Force Manu- 
als, vahd small data base, management, for example, management of 
failure reports for a world-wide Air Force computer system which 
are typed in for display and reference in a manner very similar to 
the proposed handling of Datalog and other engineering informa- 
tion. Utility services normally include substantial training and 
applications consulting, which we believe gives SRI unique 
insights into the problems of implementing this project at the 
publishers and engineer *s sites. 

One NLS Utility customer, the Long Range Planning Group of Bell of 
Canada, has been particularly interested in the measurement and 
survey of NLS use. Bell recently completed a year-long detailed 
survey of the attitudes of various users towards NLS [15]. Some 
of the results may be of use in the design and interpretation of 
user surveys in this study. More directly applicable is the sys- 
tem for measuring the behavior of NLS users described under Re- 
search Design, above. The aptness of this development, which was 
sponsored by another ARC client, to a wide range of research pro- 
jects is an example of the useful synergy in the NLS community. 
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RICHARD C. HARKNESS 

POLICY ANALYST 

TELECOMMUNICATIONS SCIENCES CENTER 

INFORMATION SCIENCE AND ENGINEERING DIVISION 

Representative research assignments at SRI 

Research and assessment of telecommunications-transportation 
interactions for purposes of forming future national policy- 
Other professional experience 

Design engineer, Ocean Science and Engineering, Inc. 

Lecturer, University of Washington: course entitled 
"Innovations in Urban Transportation" 

Transportation planning engineer, Puget Sound Governmental 
Conference: worked with computer network models, evaluation of 
new transportation technologies, and developed a systems-based 
management plan for PSGC's work program 

. Marketing manager, Boeing Aerospace Company: involved in vari- 
ous staff studies, including BAC*s decision to enter ground 
tj* ah sport at ion market, telecommunications as a substitute for 
travel, the market for personal rapid transit, Saudi Arabia's 
transport needs, BAC long range planning, and international 
business strategy planning; toured the United States and Japan 
to assess business and market opportunites for PRT 

Academic background 

. B.S. in electrical engineering (1964), Duke University, M.S.E. 
in urban transportation planning (1970), and Pl.D. in urban 
system planning (1973), University of Washington 

Publications 

"Innovations in Telecommunications and Their Impact on Urban 
Life," in Innovations for New and Established Cities, G. Golay, 
ed. Praeger Publishers, Inc., New York (1975) 

"Telecommunications Substitutes for Travel: A Preliminary As- 
sessment of their Potential for Reducing Urban Transportation 
Costs by Altering Office Location Patterns," Ph.D. 
Dissertation, University of Wasington (June, 1973); also pub- 
lished by Office of Telecmmunications, U.S. Department of 
Commerce, through MTIS 
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"Communication Innovations, Urban Form and Travel Demand, Some 
Hypotheses and a Bibliography," Research Report, Urban Trans- 
portation Program Universty of Washington; updated version pub- 
lished by the Council of Planning Librarians and Trans. (1973) 

"The Transmodal Capsule Concept: An Alternative to Dual Node," 
paper presented at the 1973 Minneapolis PRT conference and pub- 
lished in conference proceedings (May 4, 1973) 
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ELIZABETH K. MICHAEL 

SENIOR RESEARCH ENGINEER 

AUGMENTATION RESEARCH CENTER 

INFORMATION SCIENCE AND ENGINEERING DIVISION 

Specialized professional competence 

Design and development of large software systems; man-machine 
system analysis; interactive and batch management; information 
management and retrieval systems; tutorial database systems; 
office automation systems; text formatting systems; distributed 
computing systems 

Representative research assignment at SRI (since 1972) 

Project leader for an extensive study in the area of informa- 
tion processing and documentation production. Study included 
online delivery and retrieval systems. 

Program coordinator for the integration of a very large, 
interactive application system into a multi-computer network 
environment . 

. Extensive client consulting in areas of information systems, 
and large scale technical document production. 

. Design of information retrieval and tutorial help systems. 

Development of the computer system and associated organization- 
al techniques at the Augmentation Research Center. 

Other experience 

Stanford Computation Center, Stanford University; design, im- 
plementation, and administration of system measurement and user 
accounting system for an IBM 360/70 computer supporting 
interactive terminals simultaneously with multiple batch 
processes 

. Manager, Programming and Analysis, Controllers Office, Stanford 
University; supervision and training of programmer analysts; 
design and implementation of all University administrative 
computerized tasks including student registration and records, 
alumni information, business, and accounting 
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Academic background 

B.S. in physical science/organic chemistry (1947) and M.A. in 
economic statistics (1948), Stanford University 

Publications 

Co-author of the Document Production and Control System Design 
Study 

Various articles and manuals for SRI's Augmentation Research 
Center and the Stanford Computation Center; articles on comput- 
er measurement and accounting for the SHARE Computer Measure- 
ment and Evaluation Project 
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DIRK H. van NOUHUYS 

RESEARCH ANALYST 

AUGMENTATION RESEARCH CENTER 

INFORMATION SCIENCE AND ENGINEERING DIVISION 

Specialized professional competence 

Technical writing; all types of writing; technical publications 
management; application of computer-based techniques to techni- 
cal publications; teaching writing 

Representative research assignments at SRI (since 1970) 

Technical writer: developed an advanced, computer-based 
interactive text handling and information retrieval system; or- 
ganized and participated in online composition and printing of 
reports and proposals; participated in design and debugging of 
the command language; taught and developed tutorial materials 
for the online language; developed and operated interactive re- 
trieval and cataloguing systems; coordinated and promoted de- 
velopment of computer-based text processing systems 

Other professional experience 

Management of the Resource Data Center, TRW Systems (Redondo 
Beach): wrote proposals 

. Technical writer and editor, Western Regional Research Labora- 
tory of the US DA (Albany) 

Academic background 

. B.A. in writing (1956), Stanford University; M.A. in 

contemporary literature (1957), Columbia University; additional 
study in physics, psychology, French, and English 

Publications 

. Coauthor of three SRI technical reports on computer system de- 
velopment of text processing systems, training in use of online 
computer systems, and interactive question answering systems; 
coauthor of four technical movies or Video tapes 

Professional society 

Society of Technical Communication 
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JAMES H. BAIR, JR. 

SENIOR INFORMATION SCIENTIST 

AUGMENTATION RESEARCH CENTER 

INFORMATION SCIENCE AND ENGINEERING DIVISION 

Specialized professional competence 

Man-computer systems development and analysis; information sys- 
tem design and operation; psychology of human communication and 
group dynamics; organizational behavior research; teaching 

Representative research assignments at SRI 

Senior Information Scientist: supervisor, Applications Devel- 
opment Group; leader, technology transfer program analysis; 
courseware design and user training for the Online System 
(NLS); analysis of man-computer interaction systems; coordina- 
tion of user-system documentation; leader, ARC Seminar Program 

Other professional experience 

Rome Air Development Center: Project Officer — Behavioral 
Scientist; evaluation and analysis of computer text processing 
systems (task leader); USAF Coordinator for R&D in man-computer 
communication; project monitoring for system development-; re- 
search in human information processing 

College instruction, Syracuse University, Mohawk Valley 
College, and Pennsylvania State University: conducted courses 
in social psychology, group dynamics, and human communication 

Academic background 

. B.S., mass communication (1965), Utah State University; M.A., 
psychology of human communication systems (1967), Pennsylvania 
State University 

Most recent publications and reports 

"The Future Designers of Computer Systems: Social 
Scientists?," Telecommunications Policy, Spring, 1977 (in 
press) 

"Strategies for the Human Use of Computer Based Systems," NATO 
Advanced Studies Institute on Man Computer Interaction, Mati, 
Greece, Sept. 1976 (Invited Lecture; to be published in NATO 
Proceedings) 
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. Coauthor with Jacque Vallee, R. Johansen, & G. Edwards, "A Time 
Series Approach to the Evaluation of Office Automation," 
Invited Paper for the XEROX Corp. Workshop on the Evaluation of 
Office Automation, May 1976 

"Technology in Instruction and Education," In: Technology 
Transfer: A State of the Art Survey, T. Anyos and K. 
Hirschberg (editors), Stanford Research Institute Final Report, 
1975, pp. 103-110 

Professional association 

Chairman, Human-Computer Communication Working Group, 
International Federation for Information Processing; American 
Society for Information Science (Session Chairman, Representa- 
tive); International Communication Association (Officer, 
Session Chairman, Symposium Chairman); Speech Communication 
Assoc, of America (speaker); IEEE (Speaker); Human Factors 
Society. 
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HARVEY G. LEHTMAN 

RESEARCH ENGINEER 

AUGMENTATION RESEARCH CENTER 

INFORMATION SCIENCE AND ENGINEERING DIVISION 

Specialized professional competence 

Design and development of large software systems; man-machine 
system analysis; software engineering tools and techniques; 
tutorial database systems; office automation systems; text 
formatting systems; distributed computing systems 

Representative research assignment at SRI (since 1970) 

Development of the computer system and associated organization- 
al techniques at the Augmentation Research Center 

Software engineering: tools and techniques for aiding in the 
development of large software systems 

Development of information retrieval and tutorial help systems 

Two-dimensional display terminal formatting 
. Random file system revision: .addition of property lists 

Distributed operating system version of NLS 

Report writing; languages and project reports 

Other experience 

Teaching assistant in computer science and programming, 
University of California, San Diego 

Research in experimental high energy physics. University of 
California, Berkeley and University of Chicago 

Academic background 

. B.A. in physics (with honors, 1966), University of California, 
Berkeley; M.S. in physics (1967), University of Chicago; work 
toward Ph.D. in physics (through 1969), University of Chicago; 
graduate work in Computer Science (1970), University of 
California, San Diego; MS in Engineering-Economic Systems 
(1976), Stanford University; current Ph.D. candidate in Indus- 
trial Engineering with specialization in man-machine systems, 
Stanford University 
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DAVID E. RAPHAEL 

PROGRAM MANAGER AND SENIOR INDUSTRIAL ECONOMIST 

BUSNESS INTELLIGENCE PROGRAM 

OFFICE OF RESEARCH OPERATIONS 

Specialized professional competence 

Economic studies in aviation, travel, personal transportation, 
international economics indicators, telecommunications, elec- 
tronics funds transfer systems, computer security, automated 
personal identification, personal privacy, nuclear energy, 
aerospace market diversification, country studies; econometric 
and technological forecasting; corporate planning models 

Representative research assignments at SRI (since 1973) 

. Manager, DATALOG (a monthly publication of the Business 

Intelligence Program, which provides over 120 research docu- 
ments anually to clients on a wide range of subjects including 
global issues, industry analyses, country studies, and 
technological research) 

Program Manager, responsible to date for 

preparation/publication of 20 research documents, participation 
in five research seminars and three innovation search seminars. 
Responsible for recommending research projects in air transpor- 
tation, energy, natural resources, and financial industries 

Participated in SRI contract research projects in aviation, 
travel vs. telecommunications, market diversification 
opportunities, and projects requiring international economic 
forecasts 

Other professional experience 

Corporate planning manager, McDonnell Douglas corporate head- 
quarters; managed the corporate planning activities regarding 
commercial aviation/diversification; participated in the prepa- 
ration of corporate long range plans, analyses of the future 
business environment; corporate economist; reported to the se- 
nior vice president and other top management executives on a 
number of special projects, one of which resulted in the 
creation of a successful $50 million firm 

Project leader and nuclear design physicist, Universty of 
California Lawrence Livermore Laboratories; responsible for the 
nuclear physics design of several major missile warhead systems 
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presently in the U.S. inventory; AEC physics advisor on 
atmospheric and underground nuclear explosions 

Division Officer, U.S. Naval Air Command, Pacific Fleet; air- 
craft carrier, U.S.S. Hornet (CVA-12), officer of the deck 
(under way); Lieutenant Commander, USNR 

Instructor on graduate economics, Golden Gate University/ACE 
Stanford University 

Academic background 

. B.S. in physics, University of San Francisco; graduate courses 
in physics, University of California; MBA in business 
economics, St. Louis University, Ph.D. candidate in 
international economics and finance, St. Louis University 

Publications 

"World Commerical Airline Industry," "Travel vs. 
Telecommunications," "Whither Business Planning Models," 
"Global Economics Prospects," "Global Inflation," 
"Telecommunications Services Industry," "Automated Personal 
Identification," "U.S. Defense Industry" .. 

. Manager, INTERNATIONAL ECONOMIC INDICATORS, "Nuclear Power 
Prospects," "Electronics Funds Transfer," " Electric Power — 
Cost to Industry," "Preserving Personal Privacy in Business," 
"Personal Transportation," MEXICO, "Fire Prevention and Con- 
trol," "Industrial Noise Control," "Capital Availability," 
"Natural Gas in Western Europe" 

Professional associations 

National Association of Business Economists; Commonwealth Club; 
World Affairs Council 

Participated in SRI contract research projects in aviation, 
travel vs. telecommunications, market diversification 
opportunities, and projects requiring international economic 
forecasts 
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DAVID A. POTTER 

SENIOR INFORMATION SCIENTIST 

AUGMENTATION RESEARCH CENTER 

INFORMATION SCIENCE AND ENGINEERING DIVISION 

Specialized professional competence 

Planning, implementing, and managing all phases of social 
science research projects, from proposal writing through data 
analysis and report writing. 

Representative research assignment at SRI 

Planned, coordinated, and controlled marketing efforts of the 
Augmentation Research Center. This involved the simultaneous 
development of all phases of a marketing plan and strategy for 
NLS, including the development of a marketing information man- 
agement system, the cultivation of a group of prospects, direct 
client and prospect contacts and system demonstrations, and 
general strategy planning. 

Other professional experience 

Research Psychologist, Educational Testing Service, Teacher 
Behavior Research Group, Princeton, New Jersey 

. Supervisor, Organizational and Personnel Consulting, Ernst & 
Ernst, St. Louis, Missouri 

Instructor, Summer Session, Department of Psychology, State 
University of New York, College at Cortland, New York 

Research Assistant, Department of Psychology, Cornell 
University, Ithaca, New York 

. Lecturer, Department of Psychology, University of Delaware, 
Newark, Delaware 

Teaching Fellow, Department of Psychology, Cornell University, 
Ithaca, New York 

Academic background 

B.A. in Psychology, University of Delaware, Newark, Delaware; 
M.A. in Social Psychology and Industrial Psychology, Cornell 
University; Ph.D. in Personality and Social Psychology, Cornell 
University 



Part One—Technical Proposal page 50 



EKM JHB RCH DVN 10 APR 77 29248 
Innovations in Engineering Publications 



Most recent publications 

"Honest, Useful, and Unpretentious," Review of W.S. MacDonald 
and G. Tanabe, Focus on Classroom Behavior: Readings and Re- 
search, Contemporary Psychology, 1975, Vol. 20, 51-52 

Coathor of "NLS Application Description: Educational Testing 
Service," Menlo Park, California: Stanford Research Institute 
(NLS Journal 33201), 1975 

"Job Analysis of Teaching," Paper presented as part of a 
symposium at the annual convention of the American Educational 
Research Association, San Francisco, April 1976 

"The Automated Researcher: A Total Systems Approach to Knowl- 
edge Work," Paper presented at the annual convention of the 
American Educational Research Association, San Francisco, April 
1976 

"Recommendations for a Teacher Licensing Examination," Paper 
commissioned by the Commissioner's Task Force on Teacher Certi- 
fication, New York State, 1976 
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JEANNE M. BECK 

RESEARCH ASSOCIATE 

AUGMENTATION RESEARCH CENTER 

INFORMATION SCIENCE AND ENGINEERING DIVISION 

Specialized professional competence 

Technical writing and editing; documentation of computer sys- 
tems, software, and hardware; development of instructional ma- 
terials and strategies; problem-solving and traning people in 
the use of computer systems 

Representative research assignments at SRI (since 1973) 

Writing of user and system documentation for the development of 
an advanced, computer-based interactive text-handling and in- 
formation retrieval system; participation in online composition 
and printing of reports and proposals 

Development, writing, and operation of an interactive computer 
help system 

Direct on-site assistance at ARPA (Advanced Research Projects 
Agency): office systems analysis, consulting, documentation, 
and' training in the use of computer systems 

. Teaching and development of tutorial materials for the online 
language 

Training of user and integration of computer-based services 
into working environments 

Academic Background 

. B.A. in Futures Research (with honors, 1974), California State 
College, Sonoma 
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COMMITMENTS OF PRINCIPAL STAFF 



Elizabeth K. Michael, the Principal Investigator, is currently 
committed to the following contracts: 

The Document Production and Control System Design Study (DPCSDS). 
This project is funded by Rome Air Development Command (RADC) for 
$100,000. The contract runs from June 15, 1976 through September 
15, 1977. The annual budget is approximately $80,000. 

The EPA Foundry Project. This project is funded by the Environ- 
mental Protection Agency for $15,000. It is a subcontract for 
processing technical information on foundries. The main contract 
is an economic impact study of pollution controls. The contract 
dates are April 1, 1977 through April 1, 1978. 

Proposed contract: 

Consulting services for the Air Force Data Systems Design Center. 
This contract will be funded by Rome Air Development Command for 
$14,931 and will run for 15 weeks beginning this fiscal year. The 
exact number of person-months that Ms. Michael will devote to this 
project is unknown, but the support will only be occassional. 



Other research personnel to be involved: 

Dirk H. van Nouhuys is currently committed to the following two 
contracts : 

The Document Production and Control System Design Study (DPCSDS). 
This project is funded by Rome Air Development Command (RADC) for 
$100,000. The contract runs from June 15, 1976 through September 
15, 1977. The annual budget is approximately $80,000. He has al- 
ready devoted five person-months to this contract and will devote 
about two more. 

Workshop Utility Service IV — Applications Services. This contract 
deals with customer support functions for the NLS Utility 
Workshop. Mr. van Nouhuys is committed only on an as-needed 
basis, which is less than half time. It is funded by Utility 
clients for $598,781 from January 18, 1977 through October 1, 
1977. The annual budget is $798,372. 

Richard Harkness is currently committed to the following contract: 
A Technology Assessment of Telecommunication Interactions With 
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Transportation. Dr. Harkness is the Principal Investigator for 
this contract, a technical assessment which deals with the possi- 
bility for teleconferencing and office automative technology to 
restructure organizations in a way that will reduce travel de- 
mands. This interdisciplinary study is funded by NSF-RANN for 
$280,000. The study will be completed upon the submission of the 
final report in mid-May. 

James H. Bair is currently committed to the following contract: 

Workshop Utility Service IV — Applications Development. This con- 
tract deals with customer support functions for the NLS Utility 
Workshop. Mr. Bair has committed approximately three fourths of 
his time to this work through October 1 . It is funded by Utility 
clients for $598,781 from January 18, 1977 through October 1, 
1977. The annual budget is $798,372. 
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APPENDIX A Job Task Types 

The following classification of job task types are similar to 
those used in an earlier study [7]. 

Note that the percentage of time spent in task type 8 may be 
strongly influenced by the experiment. This assessment of time 
allocation will also be requested during the posttest. 

1 . Hardware or software work 

2. Project engineering, including: contract paperwork (forms 
memos, etc.), reviewing proposals and reports 

3. Writing plans and/or reports 

4. Software or hardware operation (including evaluation, 
debugging of software packages) 

5. Briefings and/or demonstrations of systems 

6. Managing other personnel 

7. Administrative paperwork 

8. Study, review of the state-of-the-art, reading, literature 
search, review of professional news 

9. Clerical work (typing, reproduction, errands, etc.) 

Additional data will include job title, organizational role (e.g., 
manager, project engineer), length of employment, years in the 
profession, experience with computers, skills posessed, number of 
persons supervised, and amount of funds controlled. This data 
will permit a discussion of information seeking behavior and how 
it relates to the experiment device. [2] 
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APPENDIX B Areas to Be Explored in Attitude Questionnaires and 
Surveys 

One goal of the questionnaires and surveys is to extract attitudes 
toward technology and STI in general, and the service provided in 
particular. The following lists give the flavor or the informa- 
tion we hope to obtain. 

Attitudes toward technology and STI: 

1 . automation 

2. automated libraries 

3. feeling toward computers 

4. confidence in computers 

5. reliability of computers 

6. advantages of computer based information services 

7. personal access to a computer 

8. personal operation of a terminal 

9. online retrieval of STI 

10. typing on a computer terminal 

1 1 . formulating STI retrieval queries 

12. acceptability of terminals for service by colleagues 

13. requirements for skill to use a terminal 

14. information storage and retrieval systems 
• 15. selective dlssemation systems 

16. SDI Profiles 

17. using keywords to indicate desired items 

18. information overload 

19. availability of STI 

20. usefulness of STI 

21. usefulness of current technical library 

22. desirability of economic and marketing information 

Attitudes toward the service provided: 

1. relevance of items retrieved (relevance metric) by experimental 
service 

2. time from publication to review by subject (timeliness metric) 

3. time spent reviewing STI (review-t metric) 

4. use of manual library services 

5. sources of STI (library, colleague, formal, informal, etc.) 

6. percent of time spent reviewing STI 

7. number of time system access not possible 

8. problems with the terminal or service 

9. improvement in effectiveness 

10. use of the comment facility 

1 1 . training delivered 

12. training required 
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13. type of terminal used 

14. assistance required 

15. ease of learning 

16. quantity of new information discovered (per item) 

17. quantity of training received 

18. desired increase in data bases 

19. terminal design 

20. command language design 

21 . ease of use 

22. overall efficiency/productivity 

23. quality of work 

24. professional image 

25. job performance — information dependent achievements 

26. letters to the editor (pretest) 

27. usefulness of the data bases 
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